Methods and systems for time-encoded multiplexed imaging

ABSTRACT

An imaging system uses a dynamically varying coded mask, such as a spatial light modulator (SLM), to time-encode multiple degrees of freedom of a light field in parallel and a detector and processor to decode the encoded information. The encoded information may be decoded at the pixel level (e.g., with independently modulated counters in each pixel), on a read-out integrated circuit coupled to the detector, or on a circuit external to the detector. For example, the SLM, detector, and processor may create modulation sequences representing a system of linear equations where the variables represent a degree of freedom of the light field that is being sensed. If the number of equations and variables form a fully determined or overdetermined system of linear equations, the system of linear equations&#39; solution can be determined through a matrix inverse. Otherwise, a solution can be determined with compressed sensing reconstruction techniques with the constraint that the signal is sparse in the frequency domain.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the priority benefit of U.S. Application No.62/352,267, which was filed on Jun. 20, 2016, and is incorporated hereinby reference in its entirety.

GOVERNMENT SUPPORT

This invention was made with Government support under Contract No.FA8721-05-C-0002 awarded by the U.S. Air Force. The Government hascertain rights in the invention.

BACKGROUND

Wide-area motion imaging (WAMI) has received increased attention fordefense and commercial applications due to the importance of wide-areapersistent surveillance for homeland protection, battlefield situationalawareness, environmental monitoring, and intelligence, surveillance, andreconnaissance of denied areas. Recently developed systems, such asArgus-IS, can surveil up to 100 km² at over a gigapixel resolution froman airborne platform. This huge amount of visual data requiresalgorithms for automated detection and tracking of targets of interest.However, traditional kinematic data based tracking algorithms havechallenges in wide area motion imagery due to a relatively low samplingrate, low spatial resolution, occlusions, changes in lighting, andmultiple confusers. Incorporating hyperspectral data can boost theprobability of detection, reduce false alarms, and improve performancein vehicle tracking and dismount detection.

Currently fielded imaging spectrometers use either dispersive orinterferometric techniques. A dispersive spectrometer uses a grating orprism to disperse the spectrum along one axis of a focal plane array(FPA) while the other axis is used to measure a single spatialdimension. An interferometric spectrometer reconstructs the spectrumfrom an interferogram measured at the FPA by splitting the incidentlight into two optical paths and varying the optical path distance ofone of the paths with a moveable mirror.

Neither dispersive spectrometers nor interferometric spectrometers aresuitable for motion imaging a large area on the ground. For example, tocover 64 km² at a ground sampling distance of 0.5 m, an update rate of 1Hz, and up to 256 spectral bands, a dispersive grating spectrometer mustsacrifice signal-to-noise ratio (SNR) (<4 μs dwell time per pixel). Aninterferometric spectrometer is not even capable of imaging at a 1 Hzupdate rate as its mirror would have to move more than an order ofmagnitude faster (65,000 steps/sec) than what is typically available(2000 steps/sec). Given these constraints, it is not surprising that nomilitary or commercial WAMI platform has a hyperspectral sensingcapability. Therefore, today's systems can offer large area coverage orwide spectral bandwidth, but not both.

SUMMARY

Time-encoded multiplexed imaging has the potential to enable wide areahyperspectral motion imaging as it has greater throughput than adispersive imager and a faster scan rate than an interferometric imager.It can be implemented with an imaging system that includes a first lens,a spatial light modulator (SLM), a second lens, and a detector array. Inoperation, the first lens images a first point in an object plane to afirst point in a first focal plane and images a second point in theobject plane to a second point in the first plane. The SLM, which isdisposed in the first plane, encodes the first point in the first planewith a first temporal modulation and encodes the second point in thefirst plane with a second temporal modulation different from the firsttemporal modulation. The second lens, which is in optical communicationwith the SLM, images the first point in the first plane to a first pointin a second plane and the second point in the first plane to a secondpoint in the second plane. And the detector array, which is disposed inthe second plane, includes a first detector element positioned to senseboth the first temporal modulation and the second temporal modulation.

Another example imaging system includes an SLM, an optical element inoptical communication with the SLM, a detector array, and a processoroperably coupled to the detector array. The SLM temporally encodesdifferent portions of a light field with respective temporal modulationsthat are based on a Hadamard matrix. The optical element spatiallycombines the different portions of the light field at a first plane,where the detector array detects the light field at a spatial resolutionlower than a spatial resolution of the SLM. The processor samples anoutput of the detector array at a rate based on the respective temporalmodulations.

Yet another example imaging system includes an SLM, a focal plane arrayin optical communication with the SLM, and a processor operably coupledto the focal plane array. The SLM applies temporal encoding sequences tomultiple image features in parallel. The focal plane array samples thetemporal encoding sequences. And the processor produces, based on thetemporal encoding sequences, a super-resolution image, a hyperspectralimage, a polarimetric image, a plenoptic image, and/or a spatiallymultiplexed image.

It should be appreciated that all combinations of the foregoing conceptsand additional concepts discussed in greater detail below (provided suchconcepts are not mutually inconsistent) are contemplated as being partof the inventive subject matter disclosed herein. In particular, allcombinations of claimed subject matter appearing at the end of thisdisclosure are contemplated as being part of the inventive subjectmatter disclosed herein. It should also be appreciated that terminologyexplicitly employed herein that also may appear in any disclosureincorporated by reference should be accorded a meaning most consistentwith the particular concepts disclosed herein.

BRIEF DESCRIPTIONS OF THE DRAWINGS

The skilled artisan will understand that the drawings primarily are forillustrative purposes and are not intended to limit the scope of theinventive subject matter described herein. The drawings are notnecessarily to scale; in some instances, various aspects of theinventive subject matter disclosed herein may be shown exaggerated orenlarged in the drawings to facilitate an understanding of differentfeatures. In the drawings, like reference characters generally refer tolike features (e.g., functionally similar and/or structurally similarelements).

FIG. 1 shows a time-encoded, spectrally multiplexed system that performsimage encoding with a spatial light modulator (top) and image decodingwith a digital focal plane array (DFPA) (bottom).

FIG. 2 shows an alternative system for time-encoded, spectrallymultiplexed imaging.

FIG. 3 shows a time-encoded super-resolved imaging system.

FIG. 4 shows a dispersing and recombining time-encoded imaging system.

FIG. 5A shows a system with a time-encoded aperture mask proximate to apupil plane.

FIG. 5B shows a system for multiplexed ray angle imaging.

FIG. 5C shows a system for spatially multiplexed imaging.

FIG. 6 shows a system with multiple time-encoded aperture masks used inan imaging system with optically multiplexed fields of view.

FIG. 7 shows a dispersing time-encoded imaging system.

FIG. 8 shows a dispersing time-encoded imaging system for division ofaperture optically multiplexed imaging.

FIG. 9 shows a dispersing time-encoded imaging system with atwo-dimensional dispersing element.

FIG. 10A shows a time-encoded, polarization-multiplexed imaging system.

FIG. 10B shows images at the spatial light modulator (SLM) and focalplane array (FPA) planes in the system of FIG. 10A for different inputpolarizations.

FIG. 11A shows an alternative time-encoded, polarization-multiplexedimaging system.

FIG. 11B shows images of the polarization-dependent point spreadfunction (PSF) at the SLM plane in the system of FIG. 11A for differentinput polarizations.

FIG. 12A is a photograph of an experimental implementation of atime-encoded, spectrally multiplexed imaging system.

FIG. 12B is a plot of a 128-channel spectrum of two pixels from an imageof two LEDs with center wavelengths of 1300 nm and 1450 nm (right) andFWHM of 100 nm collected by the system shown in FIG. 12A.

FIG. 13 shows experimental spectra and images of LEDs with centerwavelengths of 1450 nm and 1600 nm and a table showing the relationshipbetween number of codes, number of frames, frame rate, hypercube rate,and spectral resolution for a time-encoded, spectrally multiplexedimaging system.

FIG. 14 is a plot of reconstructed and decoded spectra of LEDs withcenter wavelengths of 1450 nm and 1600 nm.

FIG. 15A is an experimentally acquired image of seven LEDs, each ofwhich has a different center wavelength, showing different temporalmodulation waveforms for each LED.

FIG. 15B shows a pixel with an up/down counter in a DFPA pixel used tosample the incident beams encoded with the temporal modulation of FIG.15A.

FIG. 15C illustrates images of the modulated LEDs in FIG. 15A sampledwith a DFPA with up/down counters like the one shown in FIG. 15A.

DETAILED DESCRIPTION

Time-encoded multiplexing imaging systems map different spectralfeatures, polarization features, fields of view, or ray angles in ascene to orthogonal temporal codes. This allows them to measureinformation from an observed scene more efficiently than other imagingtechnologies. Time-encoded multiplexing is useful in multi-dimensionalimaging applications, including but not limited to hyperspectralimaging, imaging polarimetry, plenoptic imaging, three-dimensional (3D)imaging, and optically multiplexed imaging.

In a conventional imaging system, a single pixel on the focal plane canmeasure only one degree of freedom of the multi-dimensional light fieldat any moment in time. For example, a conventional pixel measures onlyintegrated light intensity. Conversely, a single pixel in a time-encodedmultiplexing system can simultaneously capture multiple degrees offreedom in each measurement. As a result, a time-encoded multiplexingimaging system can operate more quickly and/or with higher imageresolution than a conventional imaging system. Operating more quicklywith little to no degradation of signal-to-noise ratio (SNR) or spatialresolution enables staring imaging systems that can capture fasttemporal phenomena and scanning imaging systems that can scan over largeareas.

Conventional imaging systems typically scan consecutively through anumber of measurements, which degrades either the temporal resolution orframe rate of the sensor. These conventional systems are challenged whenobserving moving scenes or when placed on moving platforms. Otherconventional systems disperse the degrees of freedom of a light fieldacross the detector array to simultaneously make multiple measurements.These systems suffer a loss of spatial resolution, producing an imagewith fewer pixels than the focal plane.

Conversely, time-encoded multiplexed imaging systems can measuremultiple degrees of freedom of a light field simultaneously withoutsacrificing spatial or temporal resolution. In other words, an exampletime-encoded multiplexing imaging system can acquire multidimensionaldata with both fine spatial and fine temporal resolution. The orthogonalparallelized measurement used in time-encoded multiplexed imaging offersmany benefits, including: 1) rapid simultaneous measurements of everyimaging channel (e.g., enabling higher video rates) and/or 2) higher SNRthan conventional imaging systems.

Multiple applications exist for time-encoded multiplexed imaging systemsin industrial and defense settings. In the area of hyperspectralimaging, applications include but are not limited to: precisionagriculture, biotechnology, environmental monitoring, food inspection,industrial material identification, pharmaceuticals, defense andsecurity. Other applications include plenoptic cameras (e.g. 3D facialrecognition), imaging polarimetry (e.g., remote sensing), and opticallymultiplexed imaging (e.g., extreme panoramic video).

One particular example of this technology is hyperspectral imagers fordrones. A low-flying or maneuvering drone observes a quickly movingscene. The ability to collect fast video-rate hyperspectral dataincreases the coverage rate of a drone used to identify materials in ascene. This could be used to speed up agricultural inspections or toquickly identify dangerous materials in an industrial or defenseapplication.

A Time-Encoded, Spectrally Multiplexed Imaging System

FIG. 1 illustrates an imaging system 100 that temporally encodes anddecodes different spectral features in the light field. The imagingsystem 100 includes an optical train (top of FIG. 1) that encodes thelight field and images the encoded light field from a scene 101 onto adigital focal plane array (DFPA) 140, which detects and decodes anencoded image 141 of the scene (bottom of FIG. 1).

The optical train of the imaging system 100 shown in FIG. 1 includes aprism 101, grating, or other dispersive element that spatially separatesdifferent spectral components of the light field. A spatial lightmodulator (SLM) 120 modulates the amplitudes of each spectral componentof the light field with a predetermined sequence in time. The SLM 120can be a liquid-crystal SLM that operates in a transmissive geometry(e.g., as shown in FIG. 1) or reflective geometry, a digital micromirrordevice (DMD) or deformable mirror that operates in a reflectivegeometry, a metamaterial device, shutter or light-blocking element, orany other suitable device as known in the art. It can encode signals byredirecting, blocking, or transmitting light, e.g., to produce binary(100% or 0%) modulation or partial attenuation (grayscale) modulation.It may also modulate the phase of the incident light. Phase modulationmay include a linear phase term or so-called tilt, a quadratic phaseterm or so-called defocus, or a higher order phase term.

Different wavelengths of light illuminate different regions of the SLM120, which allows multiple wavelengths to be amplitude modulated inparallel with different sequences (e.g., code 1, code 2, and code 3shown in FIG. 1). (The SLM 120 can also phase-modulate the differentwavelengths for coherent detection.) The light modulated by the SLM 120is recombined with another dispersive element (here, another prism 130)to form a subsequent image 41 that has time-encoded wavelengthinformation.

In other examples, the dispersive element(s) and SLM may be selectedand/or positioned to encode other types of components of the lightfield. For instance, the optical train may include birefringent opticsto separate and encode polarization features. Or the optical train mayinclude an SLM placed in a pupil plane to encode plenoptic ormultiplexed field-of-view (FOV) information.

An optical detector—here, a DFPA 140—converts incoming photons in theimage 141 into a digital signal. Each pixel in the DFPA 140 includes aphotodetector 142 that generates an analog photocurrent 143 whoseamplitude is proportional to the incident photon flux. Acurrent-to-frequency converter 144 coupled to the photodetector 142converts the analog photocurrent 143 in each pixel to a digital bitstream 145 (this analog-to-digital (A/D) conversion may also beperformed in the readout electronics). For practical implementations,A/D conversion at the pixel level is faster because it happens on manypixels in parallel. This allows time-encoded signals to be sampled atkilohertz to megahertz frequencies, which enables high frameratemultidimensional motion imagery without the loss of spatial resolutionsuffered by alternative methods. For more information on A/D conversionat the pixel level and DFPAs, please see U.S. Pat. Nos. 8,179,296,8,692,176, 8,605,853, and 9,270,895, each of which is incorporatedherein by reference in its entirety.

One or more (up/down) counters 146 in each pixel record usetime-modulated sampling schemes to decode and store information 147 inthe digital bit stream 145. For example, the counters 146 may sample thedigital bit stream 145 in a pattern that is the mathematical inverse ofthe modulation applied by the SLM 120. Each counter (in each pixel) maysample the bit stream 145 with a different modulation pattern, making itpossible to sense different colors (with different modulations) indifferent sections of the DFPA 140. A processing unit 150 coupled to theDFPA 140 calculates the product of the SLM and counter modulation stepsto produce a direct measurement of the encoded degree of freedom of thelight field.

This processing can be performed in electronics at the pixel level(e.g., the counters 146), in the readout electronics, on a dedicatedcircuit (e.g., the processor 150) such as an application-specificintegrated circuit (ASIC) or field-programmable gate array (FPGA), inpost processing, or some combination thereof. In-pixel processing is apowerful and efficient way to parallelize the processing and is anothercapability of the DFPA 140. For instance, the counters 146 in the DFPApixels can be modulated independently to allow simultaneous measurementof multiple signals encoded by the SLM 120.

The DFPA 140, processor 150, and/or other electronics (not shown) mayexecute the encoding and decoding process. This involves selecting themodulation patterns used by the SLM 120, DFPA 140, and processor 150along with the additional data processing steps used to recover thelight field. An example of an encoding framework may be described byapplying Hadamard or S-matrix codes in the SLM 120, DFPA 140, andprocessor 150.

Operation of a Time-Encoded, Spectrally Multiplexed Imaging System

To illustrate time-encoded multiplexed imaging, consider a singlespatial pixel. Each pixel can operate independently, so this techniquecan scale to any size array of pixels. In FIG. 1 (top), a single spatialpixel contains three spectral colors: red, green, and blue. These colorsare assigned the orthogonal codes {0,1,1}, {1,0,1}, and {1,1,0}. Thelight is dispersed through the first prism 110 onto three pixels on theSLM 120, then recombined and measured at a single pixel detector 142 inthe DFPA 140. During the integration period, the three codes aresequenced and the detector 142 makes three measurements. During thefirst time sequence (t₁), the SLM 120 is set to the first code {0,1,1},which blocks the red light so the measurement m₁ is a sum of green andblue. This is repeated for the subsequent codes for a total of threemeasurements. An estimate of the amount of red, green, and blue lightwithin the pixel can be calculated by addition or subtraction of themeasurements. For example, the blue channel is the addition of the firsttwo measurements and subtraction of the third measurement.

The image decoding can be performed independently of the measurement byreading out an image frame for each time sequence; however, the framerate of the imager (DFPA 140) limits the image decoding rate, which inturn limits the hyperspectral data (hypercube) acquisition rate. Forexample, at 100 frames/sec and 200 spectral channels, the acquisitionrate is 0.5 Hz.

Implementing decoding with the DFPA 140 enables much faster hypercubeacquisition rates because the decoding can be performed in parallel andat the same time as the measurement. In a digital focal plane array(bottom of FIG. 1), each pixel has an analog-to-digital converter (ADC)in the form of a current-to-frequency converter 144. The ADC convertsthe input photocurrent 143 into a digital pulse stream 145, and one ormore counters 146 count the number of pulses within a given integrationperiod. The magnitude of the count is proportional to the incidentphoton flux. The counters 146 can be controlled individually to countup, down, or not at all such that a duobinary {−1,0,+1} modulationsignal can be applied.

To decode the three-channel example, each of the counters 146 is set tocount up or down during the time sequences. For example, to implementthe first code at t₁, the first counter is set to count down, and thesecond and third counters are set to count up. At the end of theintegration period, each counter 146 has an estimate of itscorresponding color channel. In other words, the counters 146 storespectrally multiplexed images of the scene. This in-pixel decoding canoccur at Megahertz rates. At a rate of 1 MHz, the system 100 can acquire200 spectral channels at a rate of 5 kHz (10,000 times greater than a100 frames/sec imager).

Mathematically, the encoded light (g) can be represented as a product ofan encoding matrix (W_(E)) and a feature vector (f): g=W_(E)f, where fis an N×1 vector of the spectral channels, W_(E) is an N×N matrix witheach row corresponding to an orthogonal code, and N is the number ofspectral channels. In order to recover the original spectralinformation, g is multiplied by a decoding matrix (W_(D)): s{circumflexover (f)}=W_(D)g such that sI=W_(E)W_(D), where I is the identity matrixand s a scalar constant. For example, for a vector of length N, aHadamard matrix of rank N can be used for both W_(E) and W_(D), and s=N.In practice, it may not be practical to use a Hadamard matrix for W_(E)since it can be difficult to apply a negative modulation to light.Instead the S-matrix is used, which contains only binary values (0,+1)and is rank N−1. To convert a Hadamard matrix to a S-matrix:W_(E)=S=(1−H)/2.

More specifically, a Hadamard matrix of rank n (H_(n)) can be used torepresent the 2-dimensional wavelength and time binary encoding patternapplied by the SLM 120. A related matrix, which is also H_(n) in thisexample, represents the 2-dimensional parallelized time-encodedmodulation of the pixel in the DFPA 140. This +1,−1 modulation can beimplemented with a counter that can count up and down as explainedabove. If the incoming wavelength intensity spectrum on each pixel inthe DFPA 140, is represented as a vector Ψ, then the estimate of thewavelength spectrum, {circumflex over (Ψ)}, can be written as:

$\hat{\Psi} = {\left( \frac{1}{n} \right)H_{n}H_{n}\Psi}$

Alternatively, an S matrix of rank n (S_(n)) is used to represent the2-dimensional wavelength and time binary encoding pattern applied by thespatial light modulator. A related matrix, which is also S_(n) in thisexample, represents the 2-dimensional parallelized time encoded samplingof the pixel. Again, Ψ represents the raw system measurement (i.e., thesignal measured on each digital register in the DFPA). The measurementin each counter is scaled by a term related to the rank of the S matrix,and then offset by a term related to an non-encoded measurement to yieldan estimate of the wavelength spectrum, {circumflex over (Ψ)}. A Jmatrix (matrix of ones) is used to represent the non-encoded term whichmay be measured directly or approximated from the encoded data.

$\hat{\Psi} = {{\left( \frac{1}{2^{m\; - 2}} \right)S_{n}S_{n}\Psi} - {J\;\Psi}}$rank(S) = n = 2^(m) − 1Time-Encoded Super-Resolved Imaging Systems

Temporal encoding can also be used in super-resolution imaging. Asunderstood by those of skill in the art, super-resolution imaging refersto enhancing the (spatial) resolution of an imaging system. Asuper-resolution imager can resolve spots that are tinier than thesystem's diffraction limit, can resolve more spots than there are pixelsin the image sensor, or both.

FIG. 2 shows a time-encoded super-resolved imager 200. It includes alens 202 and a time-encoded aperture mask 220, which can be implementedwith an SLM, placed proximate to an intermediate focal plane of the lens202. Time signatures are embedded into light passing through differentregions of the time-encoded aperture mask 220. A detector 240, such as aDFPA, in the focal plane of the lens 202 detects the time signatures.The detector 240 and a processing unit 250 process the time signaturesto produce a super-resolved image of the scene observed by the imager200.

FIG. 3 shows a time-encoded imaging system 300 that encodes differentpositions in a scene with different temporal codes. In this example, thesystem 300 observes objects at positions A and B. An objective element302 forms intermediate images A′ and B′ on a time-encoded aperture mask320, such as an SLM. The spatial resolution of the time-encoded aperturemask 320 is sufficient to uniquely encode time signatures into images A′and B′. A relay element 322 reimages A′ and B′ to form the final imagesA″ and B″ on a detector 340, such as a DFPA. The spatial resolution ofthe detector 340 is insufficient to spatially resolve A″ and B″ becauseboth spots fall within a given pixel P. Time-modulated signals ofobjects A and B are measured by pixel P (e.g., as explained above withrespect to FIG. 1). The detector 340 and a processing unit 350 separatethe time-modulated signals and use knowledge of the spatial mappingbetween the time-encoded aperture mask 320 and the detector 340 toproduce a super-resolved image. In other words, the detector 340 and/orprocessor 350 recover multiple points of spatial information from eachpixel on the detector 340 using the temporal codes. The processor 350may synthesize one or more images from this spatial information.

Dispersing and Recombining Time-Encoded Imager

FIG. 4 shows a dispersing and recombining time-encoded imager 400 likethe imaging system 100 shown in FIG. 1. The imager 400 spatiallydisperses degrees of freedom of the light field prior to a coded mask(SLM 420) and then spatially recombines those degrees of freedom afterthe coded mask such that the image reaching the detector is spatiallycongruent with the observed scene. In this case information, at eachdetector pixel represents different components of the light field thatare temporally encoded with different modulation sequences.

The imager 400 observes an object at position A. Together, an objectiveelement 402 and a dispersing element 410 form an intermediate image ofthe object in which different light field components (e.g., differentwavelengths or polarizations) are spatially separated on a time-encodedaperture mask 420, such as an SLM. The time-encoded aperture mask 420encodes time signatures into the dispersed image features A₁′, A₂′, andA₃′. Light then passes through a recombining element 430 that reversesthe dispersion from the dispersing element 410. A relay element 432 thenforms an image A″ on pixel P of a detector 440 (e.g., a DFPA) that isspatially congruent with the object at position A. Time-modulatedsignals of the light field components are measured by pixel P. Aprocessing unit 450 separates the time-modulated signals.

Knowledge of the spatial dispersion at the time-encoded aperture mask420 allows the signals to be attributed to known fight field components.For example, if a wavelength spectrum is dispersed via a prism ordiffraction grating the signals associated with modulation of A₁′, A₂′,and A₃′ will represent different known wavelength regions of themulti-spectral image A″. Alternatively a polarization dispersing elementsuch as a birefringent prism may be used to disperse polarization statesof the light field and form an image of multiple polarization states ofthe object A.

In the example shown in FIG. 4, a single point object is depicted. Anextended object produces a dispersed intermediate image in which a lightfield component from one object point may be superimposed with adifferent light field component from another object point. In this case,knowledge of the spatial dispersion pattern at the time-encoded aperturemask 420 allows for encoded signals to be attributed with the properlight field component.

Plenoptic/Optically Multiplexed Time-Encoded Imaging Systems

FIG. 5A shows a plenoptic/optically multiplexed time-encoded imagingsystem 500. In this system 500, a time-encoded aperture mask 520, suchas an SLM, is placed proximate to the aperture stop or proximate to aconjugate pupil image to encode passing light with a single that may becorrelated to the pupil region from which it entered. The time-encodedaperture mask 520 encodes time signatures into light passing throughdifferent aperture regions (a.k.a. pupil regions). A lens 530 focuseslight entering pupil regions E, F, and G onto a pixel P of a detector540, such as a DFPA. A processing unit 550 coupled to the detector 540associates the signals entering pixel P with the pupil region from whichthey entered.

In plenoptic imaging, the processor 550 correlates the pupil positions(E, F, and G) and image position (P) to determine ray angles. Inoptically multiplexed imaging, the processor 550 uses the division ofaperture optical architecture pupil region information to de-multiplexdifferent imaging channels (E, F, and G).

FIG. 5B shows a system 501 for multiplexed ray angle imaging. Theobjects' ray angles, R1, R2, R3 are focused with an objective lens 501onto a microlens array 511 with a time-encoded mask 521 placed behindthe microlens array 511. The time-encoded mask 521 modulates theamplitudes of R1, R2, and R3 at positions E, F, and G, respectively. Asecond lens 531 relays the resulting modulated image to a detector 541where it is sampled at a single pixel P. A processing unit 551 coupledto the detector 541 decodes the measurement at pixel P to recover theray information of R1, R2, and R3.

Spatially Multiplexed Time-Encoded Imaging Systems

FIG. 5C shows a system 502 for spatially multiplexed time-encodedimaging. A first lens 512 images an object onto a time-encoded mask 522.The time-encoded mask 522 modulates the amplitudes of the image atdifferent positions, temporally encoded each object position resolvablewith the first lens 512 and mask 522. A second lens 532 relays theresulting modulated image to a detector 542, which has fewer sensingelements (detector pixels) than the time-encoded mask 522 has modulatingelements (modulator pixels). In other words, multiple modulator pixelson the time-encoded mask 522 are imaged onto each detector pixel in thedetector 542. A processing unit 552 coupled to the detector 542 decodesthe detector measurements to distinguish the different object featuresimaged onto each detector pixel based on the different temporalmodulations applied by the time-encoded mask 522.

Multiple Time-Encoded Aperture Masks for Optically Multiplexed Imaging

FIG. 6 shows a system 660 with multiple time-encoded aperture masks 620a and 620 b (collectively, time-encoded aperture masks 620) foroptically multiplexed imaging. Each time-encoded aperture mask 620 isplaced in an imaging channel of the optically multiplexed imaging system600, which uses a division of aperture architecture to divide theentrance pupil of the system 600 into regions E and F. (Alternatively, adivision of amplitude architecture could also be used (i.e., dividingthe transmission of the pupil rather than the area).) Prisms 630 a and630 b bend the temporally modulated beams emitted by the time-encodedaperture masks 620, e.g., as disclosed in R. H. Shepard et al., “Designarchitectures for optically multiplexed imaging,” Optics Express23:31419-36 (23 Nov. 2015), which is incorporated herein by reference inits entirety. A lens 632 images the temporally modulated beams onto adetector array 640, which is coupled to a processing unit 650 thatseparates the modulated signals into the separate imaging channels.

Temporally modulating each imaging channel embeds a true signature intoeach imaging channel such that signals in a pixel P of the detectorarray 640 can be associated with the correct imaging channel. Eachtime-encoded aperture mask 620 may be a single spatial element perchannel (such as a shutter) to encode the entire channel uniformly. Oreach aperture mask 620 may have finer spatial resolution to encodespatial information within each channel.

A Dispersing Time-Encoded Imager

FIG. 7 shows a dispersing time-encoded imager 700 that observes objectsat positions A and B. An objective element 702 produces intermediateimages A′ and B′ on a time-encoded aperture mask 720, such as an SLM,that is spatially congruent with the object. The elements of thetime-encoded aperture mask 720 encode the intermediate image with timesignatures that may be associated with the object's 2D spatialinformation. Light then passes through a dispersing element 730, such asa prism or grating, and is focused by a relay lens 732 to a detector740. At the detector 740, light field components from the objects atpositions A and B (A₁″, A₂″, and A₃″, and B₁″, B₂″, and B₃″) arespatially dispersed and superimposed on pixels P₁, P₂, P₃, and P₄. Thepixels sample the encoded time patterns and a processing unit 750coupled to the detector 740 separates the signals.

In this case, the information for spatially reconstructing the dispersedimage is encoded in the time signatures. Knowledge of the spatialdispersion pattern is used along with the observed pixel location todetermine the light field components. For example, in a multi-spectralapplication, the dispersing element 30 may disperse the multi-spectralimage A′ into the known narrow wavelength regions A₁″, A₂″, and A₃″.Wavelength information is obtained by observing the time-encodingpattern associated with object point A in three different pixels on thedetector 740.

Division of Aperture Optically Multiplexed Imaging

FIG. 8 shows a dispersing time-encoded imager 800 for division ofaperture optically multiplexed imaging. In this optical system, thedegrees of freedom of the light field are dispersed after a time-encodedaperture mask 820 and are not spatially recombined such that the imagereaching the detector 840 is spatially incongruent with the observedscene. In this case the temporally encoded information in each detectorpixel is used in conjunction with the known spatial dispersion patternto computationally reconstruct an image that is spatially congruent withthe scene that contains additional light field information.

Prisms 810 a and 810 b in the image 800 direct two fields of view (FOV 1and FOV 2) into the system 800. FOV 1 contains two objects A₁ and A₂,and FOV 2 contains two objects B₁ and B₂. An objective lens 802 formsintermediate overlapping images of FOV 1 and FOV 2 on a time-encodedaperture mask 820, such as an SLM, in which the images A₁′ and A₂′ aresuperimposed with B₁′ and B₂′, respectively. The elements of thetime-encoded aperture mask 820 encode the multiplexed intermediateimages with time signatures that are associated with the multiplexedintermediate images' 2D spatial information. The encoded light passesthrough a relay lens 822 and a dispersing element array 830 that isspatially matched to the divided pupil regions so as to disperse eachchannel differently. The relay lens 822 produces final images A₁″, A₂″,B₁″, and B₂″ on pixels P₁, P₂, P₃ of a detector array 840. The pixelssample the time-encoded patterns, and a processing unit 850 coupled tothe detector array 840 separates the signals. The processing unit 850de-multiplexes the final image by observing multiple signals in eachpixel and correlating this information with the known dispersionpattern. In this example, pixel P₂ measures a superposition ofinformation from FOV 1 and FOV 2, but this information can bedisambiguated because the signals from object A₂ and object B₁ areencoded differently by the aperture mask 820.

1-Dimensional and 2-Dimensional Dispersing Elements

The dispersing elements shown in FIGS. 1, 4, 7, and 8 may disperse thelight field in one or two dimensions. This may be done as part of adispersing and recombining arrangement (FIGS. 1 and 4) or in adispersing arrangement (FIGS. 7 and 8).

FIG. 9 shows a two-dimensionally dispersing time-encoded imager 900. Itcan encode multiple degrees of freedom simultaneously through the use aof a dispersing element that produces a 2-dimensional dispersion patternon an SLM 910. For example, wavelength information might be dispersedhorizontally on the spatial light modulator and polarization informationmight be dispersed vertically. Other combinations of light fieldcomponents in other 2-dimensional dispersion patterns can be implementedas well.

The image 900 includes a dispersing element 910, such as a diffractiveor holographic element, that disperses light from a single point A in anobject plane 901 in two transverse dimensions (e.g., x and y, where z isthe direction of propagation). A lens 902 images the dispersed light todifferent positions A₁′, A₂′, A₃′, A₁*, A₂*, and A₃* in an image plane911. Dispersing light in two dimensions allows for multiple componentsof the light field to be encoded simultaneously. For example, thedispersing element 910 may disperse light by wavelength coarsely in onedimension and finely in an orthogonal dimensions, e.g., as with avirtual image phased array (VIPA) device. Or a wavelength dispersingprism can be used to disperse the light horizontally and apolarization-dividing prism can be used to disperse the light verticallyto produce a spectral-polarimetric imager. Other combinations of lightfield components can also be dispersed; for instance, a dispersingelement array can be used to disperse channels in an opticallymultiplexed imaging application (FIG. 8) along with a wavelength orpolarization dispersing element in the orthogonal direction.

Time Encoding for Different Polarizations

FIGS. 10A and 10B illustrate a system 1000 that time encodes differentpolarizations of an incident light field 1001. It makes measurementsthat yield the first two components of the Stokes parameters, whichdescribe the polarization state of light.

FIG. 10A shows the system's components, which include a first Wollastonprism 1008, a first dispersing element (prism) 1010, a first lens 1012,an SLM 1020, a second lens 1028, a second dispersing element (prism)1030, a second Wollaston prism 1032, a third lens 1034, and an FPA 1040in optical series with each other. The first Wollaston prism 1008separates the incident light field 1001 according to its polarization,with vertically polarized light propagating at angle out of the page andhorizontally polarized light propagating at an angle into the page. Thefirst dispersing element 1010 vertically separates the differentspectral components of the vertically and horizontally polarized beams,producing spots in the plane of the SLM 1020 that are separated bypolarization state and wavelength. The SLM 1020 modulates the phaseand/or amplitude of each spot (light field component) as a function oftime using, e.g., Hadamard encoding. The second dispersing element 1030and second Wollaston prism 1032 recombine the modulated light fieldcomponents. And the FPA 1040 senses the combined intensities ofrecombined light field components.

FIG. 10B illustrates the light field for different input polarizationsat different planes within the system 1000 shown in FIG. 10A. (Theplanes in FIG. 10B are rotated about the z axis by 90 degrees withrespect to the view shown in FIG. 10A.) The top row shows the inputpolarization, the middle row shows the intensity in the plane of the SLM1020, and the bottom row shows the intensity in the plane of the FPA1040. For vertical and horizontal input polarization, a single band ofcolor appears in the plane of the SLM 1020 and a single spot appears inthe plane of the FPA 1040. But for diagonal input polarization, thefirst Wollaston prims 1008 resolves two different polarizationcomponents, which yields two bands of color in the plane of the SLM1020, with the upper band corresponding to the vertical polarizationcomponent and the lower and corresponding to the horizontal polarizationcomponent. The second dispersing element 1030 and second Wollaston prism1032 recombine these components after they have been encoded by the SLM120 to produce a single spot in the plane of the FPA 1040.

FIGS. 11A and 11B show how the system 1000 of FIGS. 10A and 10B can beextended to measure all four Stokes parameters. More specifically, FIG.11A shows the front end of a time-encoding imaging system 1100 like theone in FIG. 10A. In this case, however, the system includes fourWollaston prisms 1108 a-1108 d. Prisms 1108 b and 1108 d are rotated by45 degrees about the optic axis. Quarter-wave plates 1106 a and 1106 bare disposed in front of and aligned to the optical axes of Wollastonprisms 1108 a and 1108 d, respectively. Together, the Wollaston prismsand quarter-wave plates resolve an incident light field 1101 into apolarization-dependent point-spread function (PSF) in the plane of theSLM 1120. When the light field contains unpolarized light, thispolarization-dependent PSF includes eight spots—one pair of spots foreach Wollaston prism 1108—with each spot illuminating a different pixelon the SLM 1120. Thus, the SLM 1120 can modulate (and the system 1100can measure) each Stokes parameter of the incident light field 1101. Theback end (not shown) of the system 1100 includes a complementaryarrangement of Wollaston prisms and quarter-wave plates that recombinethe modulated beams for detection by an FPA (not shown).

FIG. 11B shows the polarization-dependent PSF at the plane of the SLM1120 in the system 1100 in for different input polarization states.Vertically (0°) polarized light produces seven spots arranged in ahorseshoe-like shape with the opening pointed downwards. Rotating thepolarization of linearly polarized light changes the orientation of thishorseshoe-like shape as shown for horizontally (180°) and diagonally(45° and 135°) polarized light. For left-hand circular (LHC) polarizedlight and right-hand circular (RHC) polarized light, only six spotsappear in the SLM plane. And for arbitrarily polarized light, eightspots of varying intensity appear in the SLM plane. The intensities ofthese eight spots can be decomposed using the other six patterns ofspots shown in FIG. 11B as a basis set to determine the Stokesparameters characterizing the arbitrarily polarized light.

As readily appreciated by those of skill in the art, the Wollastonprisms in FIGS. 10A and 11A can be replaced by other polarizationdispersing elements, including but not limited to thin-film devices,crystal optic devices, polarization gratings, and metasurfaces. Some ofthese devices, including polarization gratings and metasurfaces, may beintegrated with wavelength-dispersing elements, such as prisms,gratings, and holographic optical elements.

Experimental Demonstration of a Programmable Hyperspectral Imaging

The time-encoded multiplexed approach enables flexible encoding anddecoding. At the spatial light modulator, panchromatic operation can beenabled by fixing the mirrors, and hyperspectral resolution can bedecreased to increase hypercube acquisition. At the DFPA, selected codesor linear combinations of codes can be decoded. This capability can beuseful for decoding only spectral bands of interest or combinations ofspectral bands for spectral matched filtering. For example, for 256spectral bands approximately half are ignored due to overlap withatmospheric water absorption bands. The DFPA can selectively decode thegood bands, whereas both the dispersive and interferometric methods needto measure the entire spectrum.

FIG. 12A is a photograph of a laboratory system that demonstratesflexible encoding and decoding. The laboratory system includedcommercial of the shelf (COTS) optical elements, a digital micromirrordevice (DMD) spatial light modulator (SLM) from Texas Instruments, and acustom MIT Lincoln Laboratory 32×32 8-channel digital focal plane array.It was used to image a 1300 nm light-emitting diode (LED) and a 1450 nmLED, each having a spectral width of 100 nm.

FIG. 12B is a plot of data collected from the LEDs by the system shownin FIG. 12A. (The inset of FIG. 12B shows the LEDs that were imaged.) Itshows the entire decoded spectrum of two pixels of the image with 10 nmspectral resolution; 128 codes are used which involved acquiring 16frames, 8 codes at a time. The SLM operated at a 10 kHz modulationfrequency.

FIG. 13 shows the capability of flexible encoding and the tradeoffbetween hypercube acquisition rate and spectral resolution in the systemof FIG. 12A. In this experiment, each frame read out from the DFPAcontained eight spectral channels. Since the SLM was operating at 10kHz, the total integration time was N×100 μs, where N is the number ofspectral channels or codes. The hypercube acquisition rate was the framerate divided by the number of frames needed to acquire the fullhypercube. For example, to acquire 128 spectral channels, sixteen frameswere used where eight spectral channels are acquired per frame.Decreasing the number of spectral channels decoded increased the overallhypercube rate.

FIG. 14 shows an example of the flexible decoding enabled with asuitable DFPA. In this simulation, the DFPA decoded the top eightprincipal components. This data was read out in a single frame. Thereconstructed spectrum shows good agreement with data acquired throughfully decoding the spectrum (FIG. 13). By decoding the principalcomponents, the hypercube acquisition rate can be increased to the framerate. For example, 64 spectral channels can be acquired at 156 Hzinstead of 19.5 Hz. Furthermore, this method can be used to implementspectral matched filtering.

FIGS. 15A-15C illustrate flexible decoding for multiple LEDs, each ofwhich emits light at a different central wavelength. The system (see,e.g., FIG. 12A) disperse and modulates the beam from each LED with adifferent binary modulation as shown in FIG. 15A. Each DFPA pixelincludes an up/down counter, shown in FIG. 15B, that can be toggled tosample the recombined, modulated light field. When toggledappropriately, the up/down counter in a given pixel can be used tofilter the incident light in a way that makes it possible same a subsetof the LEDs at a given time.

FIG. 15C shows how the duty cycle of the modulation waveform at the DFPAcan be varied. For example, in the LED experiment described above, theLED is pulsed on for 1 μs whereas the pixel modulation pulse is 200 ns.Controlling the width of the modulating pulse at the DFPA enablesdecoding of linear combinations of codes. For example, in FIG. 15C, thetwo LEDs at upper left are decoded in a single channel by adding theHadamard codes. The first LED has a code of {+1,−1,+1,−1,+1,−1,+1,−1}and the second LED has a code of {+1,+1,−1,−1,+1,+1,−1,−1}. When addedtogether, this creates a new code {+2, 0, 0, −2, +2, 0, 0, −2}. This canbe implemented at the DFPA by increasing the width of the pulse from 200ns to 400 ns.

CONCLUSION

While various inventive embodiments have been described and illustratedherein, those of ordinary skill in the art will readily envision avariety of other means and/or structures for performing the functionand/or obtaining the results and/or one or more of the advantagesdescribed herein, and each of such variations and/or modifications isdeemed to be within the scope of the inventive embodiments describedherein. More generally, those skilled in the art will readily appreciatethat all parameters, dimensions, materials, and configurations describedherein are meant to be exemplary and that the actual parameters,dimensions, materials, and/or configurations will depend upon thespecific application or applications for which the inventive teachingsis/are used. Those skilled in the art will recognize, or be able toascertain using no more than routine experimentation, many equivalentsto the specific inventive embodiments described herein. It is,therefore, to be understood that the foregoing embodiments are presentedby way of example only and that, within the scope of the appended claimsand equivalents thereto, inventive embodiments may be practicedotherwise than as specifically described and claimed. Inventiveembodiments of the present disclosure are directed to each individualfeature, system, article, material, kit, and/or method described herein.In addition, any combination of two or more such features, systems,articles, materials, kits, and/or methods, if such features, systems,articles, materials, kits, and/or methods are not mutually inconsistent,is included within the inventive scope of the present disclosure.

The above-described embodiments can be implemented in any of numerousways. For example, embodiments of designing and making the technologydisclosed herein may be implemented using hardware, software or acombination thereof. When implemented in software, the software code canbe executed on any suitable processor or collection of processors,whether provided in a single computer or distributed among multiplecomputers.

The various methods or processes (e.g., of designing and making thetechnology disclosed above) outlined herein may be coded as softwarethat is executable on one or more processors that employ any one of avariety of operating systems or platforms. Additionally, such softwaremay be written using any of a number of suitable programming languagesand/or programming or scripting tools, and also may be compiled asexecutable machine language code or intermediate code that is executedon a framework or virtual machine.

In this respect, various inventive concepts may be embodied as acomputer readable storage medium (or multiple computer readable storagemedia) (e.g., a computer memory, one or more floppy discs, compactdiscs, optical discs, magnetic tapes, flash memories, circuitconfigurations in Field Programmable Gate Arrays or other semiconductordevices, or other non-transitory medium or tangible computer storagemedium) encoded with one or more programs that, when executed on one ormore computers or other processors, perform methods that implement thevarious embodiments of the invention discussed above. The computerreadable medium or media can be transportable, such that the program orprograms stored thereon can be loaded onto one or more differentcomputers or other processors to implement various aspects of thepresent invention as discussed above.

The terms “program” or “software” are used herein in a generic sense torefer to any type of computer code or set of computer-executableinstructions that can be employed to program a computer or otherprocessor to implement various aspects of embodiments as discussedabove. Additionally, it should be appreciated that according to oneaspect, one or more computer programs that when executed perform methodsof the present invention need not reside on a single computer orprocessor, but may be distributed in a modular fashion amongst a numberof different computers or processors to implement various aspects of thepresent invention.

Computer-executable instructions may be in many forms, such as programmodules, executed by one or more computers or other devices. Generally,program modules include routines, programs, objects, components, datastructures, etc. that perform particular tasks or implement particularabstract data types. Typically, the functionality of the program modulesmay be combined or distributed as desired in various embodiments.

Also, various inventive concepts may be embodied as one or more methods,of which an example has been provided. The acts performed as part of themethod may be ordered in any suitable way. Accordingly, embodiments maybe constructed in which acts are performed in an order different thanillustrated, which may include performing some acts simultaneously, eventhough shown as sequential acts in illustrative embodiments.

All definitions, as defined and used herein, should be understood tocontrol over dictionary definitions, definitions in documentsincorporated by reference, and/or ordinary meanings of the definedterms.

The indefinite articles “a” and “an,” as used herein in thespecification and in the claims, unless clearly indicated to thecontrary, should be understood to mean “at least one.”

The phrase “and/or,” as used herein in the specification and in theclaims, should be understood to mean “either or both” of the elements soconjoined, i.e., elements that are conjunctively present in some casesand disjunctively present in other cases. Multiple elements listed with“and/or” should be construed in the same fashion, i.e., “one or more” ofthe elements so conjoined. Other elements may optionally be presentother than the elements specifically identified by the “and/or” clause,whether related or unrelated to those elements specifically identified.Thus, as a non-limiting example, a reference to “A and/or B”, when usedin conjunction with open-ended language such as “comprising” can refer,in one embodiment, to A only (optionally including elements other thanB); in another embodiment, to B only (optionally including elementsother than A); in yet another embodiment, to both A and B (optionallyincluding other elements); etc.

As used herein in the specification and in the claims, “or” should beunderstood to have the same meaning as “and/or” as defined above. Forexample, when separating items in a list, “or” or “and/or” shall beinterpreted as being inclusive, i.e., the inclusion of at least one, butalso including more than one, of a number or list of elements, and,optionally, additional unlisted items. Only terms clearly indicated tothe contrary, such as “only one of” or “exactly one of,” or, when usedin the claims, “consisting of,” will refer to the inclusion of exactlyone element of a number or list of elements. In general, the term “or”as used herein shall only be interpreted as indicating exclusivealternatives (i.e. “one or the other but not both”) when preceded byterms of exclusivity, such as “either,” “one of,” “only one of,” or“exactly one of” “Consisting essentially of,” when used in the claims,shall have its ordinary meaning as used in the field of patent law.

As used herein in the specification and in the claims, the phrase “atleast one,” in reference to a list of one or more elements, should beunderstood to mean at least one element selected from any one or more ofthe elements in the list of elements, but not necessarily including atleast one of each and every element specifically listed within the listof elements and not excluding any combinations of elements in the listof elements. This definition also allows that elements may optionally bepresent other than the elements specifically identified within the listof elements to which the phrase “at least one” refers, whether relatedor unrelated to those elements specifically identified. Thus, as anon-limiting example, “at least one of A and B” (or, equivalently, “atleast one of A or B,” or, equivalently “at least one of A and/or B”) canrefer, in one embodiment, to at least one, optionally including morethan one, A, with no B present (and optionally including elements otherthan B); in another embodiment, to at least one, optionally includingmore than one, B, with no A present (and optionally including elementsother than A); in yet another embodiment, to at least one, optionallyincluding more than one, A, and at least one, optionally including morethan one, B (and optionally including other elements); etc.

In the claims, as well as in the specification above, all transitionalphrases such as “comprising,” “including,” “carrying,” “having,”“containing,” “involving,” “holding,” “composed of,” and the like are tobe understood to be open-ended, i.e., to mean including but not limitedto. Only the transitional phrases “consisting of” and “consistingessentially of” shall be closed or semi-closed transitional phrases,respectively, as set forth in the United States Patent Office Manual ofPatent Examining Procedures, Section 2111.03.

The invention claimed is:
 1. An imaging system comprising: a first lensto image a first point in an object plane to a first point in a firstfocal plane and to image a second point in the object plane to a secondpoint in the first plane; a spatial light modulator (SLM), disposed inthe first plane, to encode the first point in the first plane with afirst temporal modulation and to encode the second point in the firstplane with a second temporal modulation different from the firsttemporal modulation; a second lens, in optical communication with theSLM, to image the first point in the first plane to a first point in asecond plane and the second point in the first plane to a second pointin the second plane; and a detector array disposed in the second plane,the detector array comprising a first detector element positioned tosense both the first temporal modulation and the second temporalmodulation, wherein the SLM comprises N spatial modulation elements, thedetector array comprises M detector elements, N is a positive integer,and M is a positive integer less than N.
 2. The imaging system of claim1, wherein the first temporal modulation and the second temporalmodulation are based on a Hadamard matrix.
 3. The imaging system ofclaim 1, further comprising: a processor, operably coupled to thedetector array, to recover information about the first point in theobject plane and the second point in the object plane based on the firsttemporal modulation and the second temporal modulation.
 4. The imagingsystem of claim 3, wherein the processor is configured to sample anoutput of the first detector element at a rate based on at least one ofthe first temporal modulation or the second temporal modulation.
 5. Theimaging system of claim 3, wherein the processor is configured toproduce an image of the object plane based on the first temporalmodulation and the second temporal modulation at a spatial resolutionhigher than a spatial resolution of the detector array.
 6. The imagingsystem of claim 1, further comprising: a first dispersive element, inoptical communication with the first lens and the SLM, to disperseincident light into different spectral components, the first lensfocusing the different spectral components from a first position in theobject plane to different positions in the first plane.
 7. The imagingsystem of claim 6, further comprising: a second dispersive element, inoptical communication with the SLM and the second lens, to recombine thedifferent spectral components.
 8. The imaging system of claim 1, furthercomprising: a first polarizing element, in optical communication withthe first lens and the SLM, to disperse incident light into differentpolarization components, the first lens focusing the differentpolarization components from a first position in the object plane todifferent positions in the first plane.
 9. A method of imagingcomprising: imaging a first point in an object plane to a first point ina first plane; imaging a second point in the object plane to a secondpoint in the first plane; encoding the first point in the first planewith a first temporal modulation; encoding the second point in the firstplane with a second temporal modulation different from the firsttemporal modulation; imaging the first point in the first plane to afirst point in a second plane; imaging the second point in the firstplane to a second point in the second plane; and detecting, with asingle detector element in an array of detector elements in the secondplane, the first temporal modulation and the second temporal modulationsimultaneously.
 10. The method of claim 9, wherein the first temporalmodulation and the second temporal modulation are based on a Hadamardmatrix.
 11. The method of claim 9, further comprising: recoveringinformation about the first point in the object plane and the secondpoint in the object plane based on the first temporal modulation and thesecond temporal modulation.
 12. The method of claim 11, whereinrecovering information comprises sampling an output of the singledetector element at a rate based on at least one of the first temporalmodulation or the second temporal modulation.
 13. The method of claim11, wherein recovering information comprises producing asuper-resolution image of the object plane based on the first temporalmodulation and the second temporal modulation.
 14. The method of claim9, further comprising: dispersing incident light into different spectralcomponents, and wherein imaging the first point in the object planecomprises focusing the different spectral components from a firstposition in the object plane to different positions in the first plane.15. The method of claim 14, wherein imaging the first point in the firstplane comprises recombining the different spectral components.
 16. Themethod of claim 9, further comprising: dispersing incident light intodifferent polarization components, and wherein imaging the first pointin the object plane comprises focusing the different polarizationcomponents from a first position in the object plane to differentpositions in the first plane.
 17. An imaging system comprising: aspatial light modulator (SLM) to temporally encode different portions ofa light field with respective temporal modulations, the respectivetemporal modulations based on a Hadamard matrix; an optical element, inoptical communication with the SLM, to spatially combine the differentportions of the light field at a first plane; a detector array, disposedin the first plane, to detect the light field at a spatial resolutionlower than a spatial resolution of the SLM; and a processor, operablycoupled to the detector array, to sample an output of the detector arrayat a rate based on the respective temporal modulations.
 18. The imagingsystem of claim 17, wherein the SLM is disposed in a first image planeand the second plane is a second image plane.
 19. The imaging system ofclaim 17, further comprising: a first dispersive element, in opticalcommunication with the SLM, to form a first image of a first field ofview on the SLM, the SLM modulating the first image with a first set oftemporal modulations; and a second dispersive element, in opticalcommunication with the SLM, to form a second image of a second field ofview on the SLM, the SLM modulating the second image with a second setof temporal modulations different from the first set of temporalmodulations.
 20. The imaging system of claim 19, wherein the detectorarray comprises a single detector element configured to detector one ofthe first set of temporal modulations different and one of the secondset of temporal modulations.
 21. The imaging system of claim 19, whereinthe processor is configured to produce representations of the firstimage and the second image based on the first set of temporalmodulations and the second set of temporal modulations.
 22. An imagingsystem comprising: a spatial light modulator (SLM) configured to applytemporal encoding sequences to multiple image features in parallel; afocal plane array, in optical communication with the SLM, to sample thetemporal encoding sequences during an integration period of the focalplane array; and a processor, operably coupled to the focal plane array,to produce, based on the temporal encoding sequences, at least one of asuper-resolution image, a hyperspectral image, a polarimetric image, aplenoptic image, or a spatially multiplexed image.
 23. The imagingsystem of claim 1, wherein the first detector element is configured todecode both the first temporal modulation and the second temporalmodulation.
 24. The imaging system of claim 1, wherein the detectorarray has a readout rate lower than modulation rates of the firsttemporal modulation and the second temporal modulation.